SketchMeHow: Interactive Projection Guided Task Instruction with User Sketches

In this work, we propose an interactive general instruction framework SketchMeHow to guidance the common users complete daily tasks in real-time. contrast conventional augmented reality-based systems, proposed utilizes user sketches as system inputs acquire users’ production intentions from drawing interfaces. Given sketches, designated task can be analyzed based on subtask division and spatial localization for each task. The projector-camera is adopted projection end-users with reality technology. To verify framework, conducted two case studies of domino arrangement bento production. From our studies, systems help novice efficiently satisfaction. We believe broaden research topics sketch-based real-world applications human-computer interaction.